Skip to content

feat(WO20): exhaustive per-row tests for the eight thin odni maps (coverage depth, batch 1) - #287

Merged
ronaldtse merged 1 commit into
mainfrom
wo20-depth-odni-cyrillic
Nov 16, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
wo20-depth-odni-cyrillic

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Summary

WO20 coverage-depth, batch 1 of the corpus audit (#283): the eight thin ODNI maps. The full letter tables of the ICS 630-01 annexes are transcribed from the held source PDFs (interscript-private-references/ics-630-01/) into row-source provenance artifacts, generated into embedded tests with scripts/gen-row-tests.mjs (the #272/#281 pattern). No map rules changed — the tests pin the maps against the printed tables; divergences are findings recorded in the artifacts and below.

Audit (corpus-wide rank, tests vs encoded rule table, ascending)

Method: per map, embedded-test count vs sub-rule count; the machine-expanded CJK/jamo tables (10k+ rules) are excluded as not letter-table-shaped. Top thin maps with locally-held sources:

map tests rules source held batch
alalc-bul-Cyrl-Latn-1997 0 66 alalc later
bgnpcgn-fas-Arab-Latn-1956 0 30 not held finding
alalc-shn-Shan-Latn-2012 1 856 alalc later
alalc-tha-Thai-Latn-2011 3 1616 alalc later
alalc-mya-Mymr-Latn-2011 6 2630 alalc later
alalc-bod-Tibt-Latn-2015 5 2178 alalc later
alalc-khm-Khmr-Latn-2013 10 3473 alalc later
alalc-lao-Laoo-Latn-2012 10 2097 alalc later
icao-heb-Hebr-Latn-9303 1 116 not held finding
alalc-lif-Limb-Latn-2011 2 211 alalc later
odni-kaz-Cyrl-Latn-2015 1 92 ics-630-01 Annex Q this PR
odni-tgk-Cyrl-Latn-2015 1 87 Annex N this PR
odni-aze-Cyrl-Latn-2015 1 86 Annex P this PR
odni-tat-Cyrl-Latn-2015 1 85 Annex S this PR
odni-uig-Cyrl-Latn-2015 1 79 Annex U this PR
odni-kir-Cyrl-Latn-2015 1 78 Annex R this PR
din-jpn-Hrkt-Latn-32708-2014 19 1236 din later
alalc-mni-Mtei-Latn-2022 4 227 alalc later
ses-ara-Arab-Latn-1930 2 93 not held finding
odni-che-Cyrl-Latn-2015 2 90 Annex C this PR
iso-tha-Thai-Latn-11940-1998 2 86 iso (TR 11941) later
bgnpcgn-arm-Armn-Latn-1981 2 78 not held finding
alalc-sat-Olck-Latn-2017 1 34 alalc later
bgnpcgn-aze-Cyrl-Latn-1993 2 64 not held finding
odni-mkd-Cyrl-Latn-2015 2 63 Annex H this PR
din-*-*-1460-2-2011 (62 maps) 4 ~45 din next batch
iso-rus-Cyrl-Latn-9-1995 5 256 iso (ISO 9) later

Per-map counts (transcribed = generated; every generated test passes)

map transcribed/generated before → after skipped (why)
odni-kaz 39 1 → 40 я (printed cell "Ya", capital-Y case slip — finding), ъ/ь (notes rows, not table rows)
odni-aze 38 1 → 39 —
odni-kir 34 1 → 35 —
odni-tat 37 1 → 38 —
odni-tgk 33 1 → 34 е (note-1 ye/e context), ъ (blank instruction), ц/щ/ь/ы ("" instruction cells)
odni-uig 35 1 → 36 ъ/ь (notes rows)
odni-che 38 2 → 40 е, ъ (context-dependent footnotes), пӏ, цӏ, ц, 1 (map divergences — findings below), ь (empty output)
odni-mkd 60 2 → 62 ’ ("[Not represented]"), Ѕ/ѕ (map divergence — finding below)

Multi-key value cells ("и, і"; "г, ҝ"; "й, ј"; "у, ү"; "|, ӏ, 1"; mkd case-pair cells) are transcribed as one row per printed key. Every value cell was verified against the rendered page image; the annexes' text layers store lookalike Cyrillic glyphs for printed Latin e/x, documented per artifact.

Findings (source-anchored; maps untouched this round)

  • odni-mkd Ѕ/ѕ (dze): the annex's letter cell prints "S s" (substitute-glyph rendering); the map encoded it literally — Latin S/s (U+0053/U+0073) → Dz/dz — while the actual Cyrillic Ѕ/ѕ (U+0405/U+0455) carry no rule and pass through. Collateral: Latin-letter input is silently rewritten to Dz/dz, against the annex's own as-is convention for Latin names.
  • odni-che ц: the printed row ц → ts has no map rule at all (plain U+0446 passes through; only single-codepoint ҹ U+04B9 is encoded). Also цӏ: printed key ц+palochka (U+0446+U+04CF) unencoded (falls through to "ts'").
  • odni-che пӏ: printed p; map lacks the lowercase digraph (falls through to "p'"); uppercase Пӏ → "Ph" is supported by no printed cell.
  • odni-che footnotes 1/3 not implemented: the map applies ъ → ' unconditionally (footnote 3 drops word-initial ъ) and word-initial Ye/ye for е (footnote 1's printed wording differs).
  • odni-che palochka-cell "1": the 1 → ӏ normalization rules don't fire at string edges — standalone "1" passes through.
  • odni-che uppercase Ы carries two sub rules (Y, then Cyrillic Е); observation recorded, untouched.
  • odni-kaz я: printed value "Ya" (capital Y) — annex case slip vs map "ya".
  • odni-tgk е: note-1 (ye word-initial / e elsewhere) not implemented (unconditional e); odni-tgk ц/щ/ы: map carries Ts/Shch/Y though the annex defers those cells to the Russian IC standard.

Gates

Test plan

  • every generated row passes end-to-end through iscrs transliterate before being written into the artifact
  • iscrs test-all green on the branch corpus (11,985/0)
  • golden additivity: 0 lost / 0 changed rows
  • ts parse gate green (435/435)
  • skipped rows + findings recorded in each scripts/row-sources/*.json note

…verage depth)

The full letter tables of ICS 630-01 Annexes Q/P/R/S/N/U/C/H are
transcribed from the held source PDFs into row-source provenance
artifacts and generated into embedded tests with
scripts/gen-row-tests.mjs. No map rules changed.

314 rows transcribed, all passing end-to-end; 15 skipped printed rows
and 11 source-anchored findings documented in the artifacts (kaz
"Ya" case slip; tgk е footnote semantics + see-Russian cells; che
palochka findings incl. missing plain-ц rule and unimplemented
footnotes 1/3; mkd dze: Latin S/s encoded instead of Ѕ/ѕ).
@ronaldtse
ronaldtse merged commit 1a7e886 into main Nov 16, 2026
13 checks passed
ronaldtse added a commit that referenced this pull request Nov 16, 2026
Three batches from held sources (maps #287/#288/#289): eight odni
ICS-630 maps, eight thinnest DIN 1460-2 maps, six small ALA-LC tables
- 906 exhaustive per-row tests, all verified end-to-end; 48
source-anchored findings recorded for follow-up fixes. Golden
refreshed additively (rs #30): 13,745 rows, zero lost/changed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant